Apparatus and method for contactless input

ABSTRACT

An apparatus for contactless input  10  having an optical image formation means  13  having cross disposed first and second reflective surfaces  20, 21  arranged numerously and standing on a same planar surface, a display  11  provided on one side of the optical image formation means  13 , and a light sensor  26  detecting only light from the front side and arranged side by side, wherein a first real image  12  is formed on the other side of the optical image formation means  13  by the optical image formation means  13  from an image  11   a  on the display  11 ; a second real image  35  is formed on the display  11  through the optical image formation means  13  from reflected light from an indicating means  34  having come into contact with the first real image  12 , and a position of the second real image  35  is detected by the light sensor  26.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a Divisional of U.S. patent application Ser.No. 15/535,924, filed Jun. 14, 2017, which is a national stageapplication filed under 35 USC 371 of International Application No.PCT/JP2015/064747, filed May 22, 2015, and which is based upon andclaims the benefit of priority from the prior Japanese PatentApplication No. 2015-027158, filed on Feb. 16, 2015, the entire contentsof which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to an apparatus and method for contactlessinput (i.e., an apparatus and method for contactlessly detecting anindicated position on a reproduced image) with and by which a real imagebecomes formed in the air, and signals can be input by operation of anindicating means (e.g., a finger) while looking at this real image(e.g., an image of a touch panel).

BACKGROUND ART

It has been known that when an image is displayed on a display and aparticular position on the image is pressed by a finger, X and Ycoordinates of the pressed part become detected by a pressure-sensitivesensor and the like, and a next behavior is performed based on thisinput signal (see, e.g., Patent Literature 1).

Also, as described in Patent Literature 2, when forming a matrix byarranging, right above a display, a number of light-emitting elementsalong X-axis and in parallel with one another and a number oflight-receiving elements along Y-axis and in parallel with one another,and touching the surface of the display with an obstacle such as afinger and a pen, it is proposed to detect a position having come intodirect contact with the display by crossing of the obstacle on thematrix.

On the other hand, in Patent Literature 3, there is proposed a methodand apparatus that detect an indicated position on a reproduced image byusing a optical image formation means in which a first light controlpanel having a number of first planar light-reflective portions arrangedin parallel with one another at constant intervals inside a transparentflat plate and a second light control panel having a number of secondplanar light-reflective portions arranged in parallel with one anotherat constant intervals inside a transparent flat plate are arranged indirect contact with or proximity to each other in a manner that makesthe first planar light-reflective portions and the second planarlight-reflective portions orthogonal in planar view, simultaneouslydisplaying an image on a display and an image formed by making infraredlight irregularly reflect off the surface of the display as reproducedimages in the air, and detecting a position of an indicating meanshaving touched the reproduced images by a two-dimensional infraredcamera.

As described in Patent Literature 4, there is also a proposal of anapparatus that cognizes a form of movement of a multi-touch by means offingers or a stylus pen on a liquid-crystal surface throughincorporating a light sensor in a transistor-shaped surface composing aliquid-crystal panel.

CITATION LIST Patent Literature

Patent Literature 1: Japanese Unexamined Patent Application PublicationNo. 2006-039745

Patent Literature 2: Japanese Unexamined Patent Application PublicationNo. 2000-056928

Patent Literature 3: Japanese Patent No. 5509391

Patent Literature 4: Japanese Unexamined Patent Application PublicationNo. 2011-029919

SUMMARY OF INVENTION Technical Problem

However, in the case of the touch panels described in Patent Literatures1 and 2, a planar display existed, a particular planar image becamedisplayed on the display, and detection of an input position wasachieved by pressing a particular position on the display. Therefore,when the image is pressed by a finger, a pen and the like, the finger,the pen and the like necessarily touch or run into the display surfaceor the touch-panel surface, and sometimes the display and the like hasbecome dirty or the display has gotten scratched.

In Patent Literature 3, besides a display, an infrared light generatingmeans (a light of infrared rays), an irregular reflection surface forinfrared light and an infrared camera are also required, which addsadditional complexity to a structure of an apparatus. Also, since theinfrared light generating means and the infrared camera are arranged atpositions different from the display and the optical image formationmeans, there has been a problem in that installation spaces becomerequired. Additionally, since the irregular reflection surface forinfrared light becomes arranged on the front side of the display, therehas been a problem in that part of light of the display becomesabsorbed.

Moreover, in Patent Literature 3, since the optical image formationmeans is used, there is no focal point distance for image formationmeans unlike a lens, however, in a case of using the infrared camera itis necessary to set the camera's position and angle and to adjust thecamera's focus on an image.

In Patent Literature 4, there is proposed an optical touch panelprovided with a liquid-crystal panel having a backlight and alsoprovided with a light sensor detecting touches on the liquid-crystalpanel by reflected light. However, the touch panel is not a type thatbecomes formed an image in the air.

Touch panels for which displays are employed are used for ATMs and thelike, however, since a large, indefinite number of people touch screens,it is not hygienic, and it has not been effective for the prevention ofcontact infections. Also, when light becomes irradiated toward thedisplays, light reflected off the displays becomes emitted from thedisplays, which sometimes made it hard to look at the displays.

The present invention has been made in view of the above circumstances,and an object thereof is to provide an apparatus and method forcontactless input that make an image to be formed a spatial image thatis not reflected light from other light sources, detect a positionindicated by an indicating means when a particular position on thisspatial image is indicated by the indicating means such as a finger, apointer, and a stylus pen, and that enable signal inputs withoutphysically touching a display.

Solution to Problem

In order to achieve the above object, according to the presentinvention, there is provided an apparatus for contactless input,comprising;

an optical image formation means having first minute reflective surfacesand second minute reflective surfaces crossed in planar view, the firstminute reflective surfaces arranged numerously and standing on a sameplanar surface, the second minute reflective surfaces arrangednumerously and standing on a same planar surface, the optical imageformation means forming a second reflected light by receiving a firstreflected light from each of the first minute reflective surfaces on thecorresponding second minute reflective surfaces, a display provided onone side of and at a distance from the optical image formation means,and a light sensor arranged on the surface of the display, the lightsensor having sensor elements detecting only light from the front side,the sensor elements provided side by side with one another, wherein afirst real image is formed on the other side of the optical imageformation means by the optical image formation means from an image onthe display; a second real image is formed on the display through theoptical image formation means from reflected light from an indicatingmeans having come into contact with the first real image, and a positionof the second real image is detected by the light sensor.

In the case of the apparatus for contactless input according to thepresent invention, it is preferred that part or all light emitted fromthe display be high-frequency-modulated (e.g., several kHz to 200 MHz).

Also, it is preferred that light emitted from the display containinfrared light, and that each of the sensor elements be an infraredsensor element. Here, it is preferred that the light sensor (an infraredsensor) be formed in the shape of a sheet.

In the case of the apparatus for contactless input according to thepresent invention, it is preferred that the back side of each of thesensor elements be provided with a non-transmissive material. When thesensor elements are light sensor elements of visible light,non-transmissive materials that block visible light are used, and whenthe sensor elements are infrared sensor elements, non-transmissivematerials that block infrared light are used.

Additionally, in order to achieve the above object, there is provided amethod for contactless input, comprising:

using (1) a optical image formation means having first minute reflectivesurfaces and second minute reflective surfaces crossed in planar view,the first minute reflective surfaces arranged numerously and standing ona same planar surface, the second minute reflective surfaces arrangednumerously and standing on a same planar surface, the optical imageformation means forming a second reflected light by receiving a firstreflected light from each of the first minute reflective surfaces on thecorresponding second minute reflective surfaces; (2) a display providedon one side of and at a distance from the optical image formation means;and (3) a light sensor provided on the surface of the display, the lightsensor having sensor elements detecting (only) light from the front sideand arranged side by side with one another;

the method for contactless input, comprising steps of:

forming a first real image on the other side of the optical imageformation means from an image on the display;

forming a second real image on the surface of the display from an imageof an indicating means having come into contact with the first realimage; and

optically detecting a position of the second real image by the lightsensor.

In the case of the method for contactless input according to the presentinvention, it is preferred that light emitted from the display containinfrared light, and that each of the sensor elements be an infraredsensor element.

Advantageous Effects of Invention

Since the apparatus and method for contactless input according to thepresent invention have, on the surface of the display, the light sensorin which the sensor elements detecting only light from the front sideare arranged side by side with one another, form a second real image onthe display through the optical image formation means by reflected lightfrom an indicating means, and detect a position of the second real imageby the light sensor, a position of the indicating means can be detectedrelatively easily without providing a special infrared light producingmeans, infrared camera and the like.

Also, since the optical image formation means does not have a specialfocal point distance unlike a lens (or a camera), even when a positionof the display is changed, an image of an indicating means becomesformed on the original display, and a position of the indicating means(e.g., a finger, a pen tip, etc.) can be detected more accurately.

Especially, in the case of the apparatus and method for contactlessinput according to the present invention, when light emitted from thedisplay contains infrared light and the sensor elements of the lightsensor are infrared sensor elements, by means of infrared light thatcannot be visually confirmed, a position of an indicating means can bedetected.

Moreover, in the case of the apparatus for contactless input accordingto the present invention, when the light sensor is formed in the shapeof a sheet, it becomes easy to produce a display having a light sensor,and it becomes possible to adopt a normal display at the same time. Whenvisible light is used as light, a conventional “liquid-crystal padhaving light sensor” can be used for the display without any change. Inthis case, it is desirable to arrange the non-transmissive materials onthe back side of the sensor elements to make the sensor elements detectonly light from the front side. In a case of putting a sheet-like lightsensor on a display, the light sensor can be provided separately fromthe display.

Additionally, the position of the indicating means (e.g., a tip offinger) is detected by arithmetic processing such as finding a centerposition of gravity by light reception data of sensor elements which isdata of the second real image received by the sheet-like light sensor.

Here, when an image on the display is a keyboard and the like, it ispreferred that a special light image (e.g., a spotlight) be provided atthe center of each keyboard key. A position of the light sensor canthereby be accorded with these radial images, which enhances detectionaccuracy.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory diagram of an apparatus for contactless inputaccording to one embodiment of the present invention.

FIG. 2 is an explanatory diagram of an optical image formation means foruse in the same apparatus for contactless input.

FIGS. 3(A) and 3(B) are an explanatory diagram of a display for use inthe same apparatus for contactless input.

DESCRIPTION OF EMBODIMENTS

Next, with reference to the accompanying drawings, descriptions will begiven on embodiments of the present invention.

As illustrated in FIG. 1, an apparatus for contactless input 10according to one embodiment of the present invention is provided with anoptical image formation means 13 that is formed at an angle α of 30 to60 degrees from and at a distance from a planar display 11, and thatforms a first real image 12 at a position symmetrical to an image 11 ato be displayed on the display 11 (see FIG. 2) by making light enterfrom the image 11 a on the display 11. Here, as a display, aside fromone that is flat-plate-shaped such as an ordinary liquid-crystaldisplay, a cubic one having a light source inside or one havingconcavities and convexities formed only on one side like a keyboard canalternatively be used, and depending on circumstances, one that is likea signboard (a still image display such as an electro luminescencedisplay) having a light source (a backlight) inside can also be used.Especially, in a case of still images, a display can be either cubic orplanar.

In a case where a transparent material such as transparent plastic andglass is used as a main material of the optical image formation means13, when light enters into the transparent material from the air andgoes out of the transparent material into the air, refraction due to thequality of the transparent material occurs. Thus, a position of thedisplay 11 is determined in view of a refraction angle. The position ofthe display with respect to the optical image formation means can bedetermined freely to an extent, and focusing required in the case ofusing a lens system is not necessary.

The optical image formation means 13 has, as illustrated in detail inFIG. 2, a tabular first light control panel 14 having a thickness t1(e.g., 0.1 to 5 mm) and a tabular second light control panel 15 having athickness t2 (e.g., 0.1 to 5 mm). The first light control panel 14 andthe second light control panel 15 are arranged with one side of them indirect contact with or proximity to each other. A number of belt-shapedplanar light-reflective portions 18 and a number of belt-shaped planarlight-reflective portions 19 are respectively arranged perpendicularlyto a surface on one side of the first light control panel 14 inside thefirst light control panel 14 at constant pitches (p1) andperpendicularly to a surface on one side of the second light controlpanel 15 inside the second light control panel 15 at constant pitches(p2). Here, the standing planar light-reflective portions 18 of thefirst light control panel 14 and the standing planar light-reflectiveportions 19 of the second light control panel 15 are arranged crossed(in an orthogonal state in this embodiment) in planar view.

Parts of the first and second light control panels 14 and 15 where theplanar light-reflective portions 18 and 19 do not exist are formed of atransparent material such as glass or transparent plastic. It ispreferred that these planar light-reflective portions 18 and 19 consistof metal sheets having high reflection efficiency, vapor-depositedmetal, metal sheets each having an adhesive agent layer at anintermediate part, or mirror sheets, and it is preferred that both thefront and back of the planar light-reflective portions 18 and 19 bereflective surfaces. However, the present invention also applies to acase where only one side of the planar light-reflective portions 18 and19 are reflective surfaces. A production method of the optical imageformation means 13 is described in, for example, WO 2009/131128 A1, etc.Among metals having high reflection efficiency are aluminum, silver,titanium, nickel, chromium and the like.

Normally, in view of production efficiency, it is preferable for pitchesp1 between each of the planar light-reflective portions 18 and pitchesp2 between each of the planar light-reflective portions 19 to be thesame, and it is preferable for the thickness t1 of the first lightcontrol panel 14 and the thickness t2 of the second light control panel15 to be the same. Thus, the pitches between both the planarlight-reflective portions 18 and the planar light-reflective portions 19will hereinafter be represented by p, and the thickness of the first andsecond light control panels 14 and 15 will hereinafter be represented byt.

When such the optical image formation means 13 is watched in planarviewed, as illustrated in a partially enlarged view in FIG. 1, theplanar light-reflective portions 18 and the planar light-reflectiveportions 19 cross one another and form a number of square frames. Anaspect ratio γ (height/width) of a single frame (i.e., a frame of onelayer) in this case is thickness (t)/pitch (p). The aspect ratio γ isapproximately 1 to 4.5, however, in order to obtain an even brighterfirst real image 12 by making light reflect off one of the planarlight-reflective portions 18 and one of the planar light-reflectiveportions 19 a plurality of times, it is desirable for the aspect ratio γto be 2.5 to 4.5 (more particularly, one that exceeds 3 and is 4.5 orless).

Each frame portion of the first light control panel 14 and the secondcontrol panel 15 forms each of first minute reflective surfaces 20 andeach of second minute reflective surfaces 21. The first minutereflective surfaces 20 and the second minute reflective surfaces 21 arecrossing in plan view. These first minute reflective surfaces 20 arearranged numerously and standing on a same planar surface, and thesesecond minute reflective surfaces 21 are arranged numerously andstanding on a same planar surface.

Therefore, light from the display 11 arranged on one side of the opticalimage formation means 13 reflects off each of the first minutereflective surfaces 20 of the first light control panel 14 (a firstreflected light) on the front side (the side of the display 11), andfurthermore reflects off the corresponding second minute reflectivesurfaces 21 of the second light control panel 15 (a second reflectedlight), then forms a first real image 12 on the other side of theoptical image formation means 13. This first real image 12 becomesformed in a space portion, and becomes the same in size as the image 11a formed on the display 11. Incident light and reflected light includecases where light jumps over one frame and reflects, aside from caseswhere light reflects only in one frame.

Next, with reference to FIGS. 1, 3(A) and 3(B), descriptions will begiven on the display 11 having been used for the apparatus forcontactless input 10. This display 11 is basically a liquid-crystaltype, and has a backlight 24, a liquid-crystal display 25, a sheet-likeinfrared sensor 26 that is an example of light sensors, and asuperficial transparent protection plate 28 a.

It is preferred that the backlight 24 emit visible light as well asinfrared light, and in this case, it is preferred that (part or all of)light from the backlight 24 be high-frequency-modulated. When both alight-emitting means A that emits visible light and a light-emittingmeans B that emits infrared light are provided separately to thebacklight 24, only the light-emitting means B may behigh-frequency-modulated. As the backlight 24, a light-emitting diodeand a fluorescent light can be used.

Further, the display using organic or inorganic electro-luminescence(for example) can be used instead of the liquid-crystal display.

The sheet-like infrared sensor 26 provided on the surface of the display11 has, as illustrated in FIGS. 3(A) and 3(B), a number of infraredsensor elements (an example of sensor elements) 28 arranged in a latticepattern, and conductor lines 29 and conductor lines 30 are arrangedrespectively on top of the infrared sensor 26 and underneath theinfrared sensor 26. On the back of the infrared sensor elements 28,conductive or non-conductive non-transmissive materials 27 are providedto make the infrared sensor elements 28 detect only light from the frontside. A numeral 25 a represents a transparent sheet.

The infrared sensor element 28 and the conductive lines 29 and 30 aretransparent, and allow visible light and infrared light to pass througheasily. This infrared sensor 26 reacts only to infrared light andproduces electromotive force. As infrared sensor elements, it is alsopossible to use a sensor that reacts to visible light and infrared lightand to provide a filter that allows only infrared light to pass throughat the upper portion of the sensor. One or both of the infrared sensorelements and conductor lines may be non-transparent, and in this case,it is desirable to enhance an aperture ratio of a display by narrowingthe area and width as much as possible.

The liquid-crystal display 25 has a well-known structure. One each ofcolor filters of R (red), G (green), and B (blue) that are part of avisible-light emitting portion are arranged in parallel with oneanother, one of liquid-crystal cells is provided right underneath eachof the R, G and B color filters, and visible light is emitted by meansof light from the lower portion of the backlight 24 turned on and offand brightness-controlled by the liquid-crystal cells. Therefore, bymeans of this liquid-crystal display 25, an image 11 a of apredetermined shape (e.g., a keyboard, etc.) becomes displayed.

The present invention is also applicable to those having a structure inwhich light-emitting diodes of R, G and B (or other colors) arranged inparallel are used as a display. When one in which an R light-emittingdiode, a G light-emitting diode, a B light-emitting diode, and aninfrared sensor are arranged side by side is considered as a singleblock, a display can be one in which these blocks are arranged in planestate.

Here, when visible light is used as light, and a visible-light sensor isused, an infrared blocking sheet can be provided on each of the planarlyarranged blocks.

In a case that strength of infrared light emitted from the display isnot sufficient, on the other side of the optical image formation means13, an infrared illuminator 32 that illuminates the whole of a rangewhere a first real image 12 of an image 11 a on the display 11 becomesformed can be arranged. A hood (reflector) is provided on thisilluminator 32 so that the light may not enter into the infrared sensors26 and the optical image formation means 13.

Light from the illuminator 32 is separated from natural light byperforming high frequency modulation. As a matter of course, when acontrol circuit makes a signal processing of the high frequency moduledinfrared light received by the infrared sensor, the control circuit hasan electric filter circuit that inputs only specific frequencies (highfrequencies) among light (which is converted to an electric signal) thatenters through the infrared sensor elements 28.

Next, descriptions will be given on a method for contactless input inwhich the apparatus for contactless input 10 is used.

As illustrated in FIG. 1, the optical image formation means 13 isarranged at a predetermined position, and on one side of the opticalimage formation means 13, the display 11 becomes arranged. On thisdisplay 11, an image 11 a of, for example, a touch panel becomesdisplayed, and the image 11 a can be changed to an arbitrary image by,for example, a selector switch and the like. Since the display 11 emitslight, by the emitted light passing through the optical image formationmeans 13, a first real image 12 of an image 11 a to be formed on thedisplay 11 becomes reproduced at a symmetrical position centering on theoptical image formation means 13.

In this case, the first real image 12 becomes formed in space, and evenwhen a position of the display 11 is changed, the first real image 12does not become out of focus, and a sharp image becomes reproduced. Whena finger 34 that is an example of the indicating means touches thisfirst real image 12, the finger 34 becomes illuminated with light(infrared light) from the display 11, and reflected light from thefinger 34 (an image of the finger 34) forms a second real image 35 onthe display 11 through the optical image formation means 13. Numerals r1to r4 represent reflected light rays from the finger 34.

Since this second real image 35 becomes formed clearly, a position ofthe second real image 35 is optically detected by the infrared sensor26. This enables clear recognition of a part on the first real image 12pressed by the finger 34. This second real image 35 becomes formedclearly regardless of the position of the display 11. Also, in FIG. 1, 0indicates that an indicating means (the finger 34) and a second realimage 35 is symmetrical to each other with respect to the optical imageformation means 13.

Further, when the infrared light power does not have sufficient strengthmore than a specified value, the infrared illuminator 32 is preferablyused. The position of the second real image 35 is recognized through animage processing of data obtained by detecting the location of theinfrared sensor elements 28 on the sheet-like (planate) infrared sensor26.

In the above embodiments, infrared light that cannot be visuallyconfirmed was used as light for detecting real images, however, thepresent invention also applies to a case where visible light is usedinstead. In this case, by high-frequency-modulating visible light, thevisible light can be separated from ambient light. Here, it is alsopossible to incorporate a light sensor into each of the blocks of R, G,and B light-emitting portions as visible light sensors.

It is preferred that a light-shielding member be provided on the backside of each of sensor elements in this case to detect only light fromthe front side of the display.

It is preferred that the sensor elements and conductor lines to beconnected with the sensor elements be transparent, however, the presentinvention applies to cases where sensor elements and conductive linesare non-transparent.

In a case of not using infrared light, an ordinary liquid-crystal panelcan be used, however, a group of light-emitting diodes each emittinglight of a predetermined color can also be used alternatively.

The present invention is not limited to the above embodiments, and forexample, an image on a display does not necessarily have to be a colorimage, and can also be a monochrome image.

In the case of the present invention, displays include not only thosesimply displaying images, but also illuminated or light-transmitted realimages and the like. That is, as a display, when using a phototransmitting member (a planar member, a curved member) such as anordinary signboard, it is desirable to provide a light-shielding memberon the back of each sensor element, or ones that detect only light fromthe front side (visible light or infrared light) can alternatively beused as the sensor elements.

The present invention also applies to cases where an apparatus forcontactless input is formed by combining each constructional elementdescribed above.

Further, the sensor sheet can be separated from the display or jointedto the display.

INDUSTRIAL APPLICABILITY

The apparatus and method for contactless input (the apparatus and methodfor contactlessly detecting an indicated position on a reproduced image)according to the present invention, when used for control panels of avariety of machinery, displays a reproduced image of a control panelhaving manual operation buttons (e.g., a keyboard, a touch panel) inspace, and is capable of obtaining input signals when the manualoperation buttons in the reproduced image become pressed. Therefore, theapparatus and method for contactless input according to the presentinvention can be used not only for control panels of factory machines,but also optimally for touch panels of mobile phones, personalcomputers, automobiles, vessels and the like.

REFERENCE SIGNS LIST

-   -   10: apparatus for contactless input, 11: display, 11 a: image,        12: first real image, 13: optical image formation means, 14:        first light control panel, 15: second light control panel, 18,        19: planar light-reflective portion, 20: first minute reflective        surface, 21: second minute reflective surface, 24: backlight,        25: liquid-crystal display, 25 a: transparent sheet, 26:        infrared sensor, 27: non-transmissive material, 28: infrared        sensor element, 28 a: transparent protection plate, 29, 30:        conductive line, 32: illuminator, 34: finger, 35: second real        image

1. A sheet-like infrared sensor sheet provided on the surface of adisplay, comprising: a plurality of infrared sensor elements arranged ina grid pattern, wherein on the back of the infrared sensor elements,non-transmissive materials are provided to make the infrared sensorelements detect only light from the front side.
 2. The infrared sensorsheet according to claim 1, wherein the display emits red, green andblue lights, and also emits infrared lights.